Information processing method, information processing apparatus and user equipment

ABSTRACT

An information processing method, an information processing apparatus and a user equipment are provided. A method comprises: acquiring sight line information of at least one user in a space; and determining, at least according to the sight line information, at least one mirror object existing in the space. Accordingly, a mirror object in a space is determined according to sight line information of at least one user, providing a basis for mirror-object-based applications, for example, modeling the mirror object.

RELATED APPLICATION

The present application claims the benefit of priority to Chinese PatentApplication No. 201510303145.3, filed on Jun. 4, 2015, and entitled“Information Processing Method, Information Processing Apparatus andUser Equipment”, which application is hereby incorporated into thepresent application by reference herein in its entirety.

TECHNICAL FIELD

The present application relates to the field of information processingtechnologies, and, for example, to an information processing method, aninformation processing apparatus and a user equipment.

BACKGROUND

In some application scenarios, modeling of an environment space isperformed. For example, in an augmented reality scenario, real objectsand virtual objects in the environment space are combined. During themodeling, if a mirror object is present in the environment space, thetraditional automatic detection method has some problems in detectingthe mirror object, for example, the mirror object can't be detected, oran image of an object in a mirror plane is considered to be a realobject.

SUMMARY

A possible objective of example embodiments of the present applicationis to provide an information processing solution.

In a first aspect, an example embodiment of the present applicationprovides an information processing method, comprising:

acquiring sight line information of at least one user in a space; and

determining, at least according to the sight line information, at leastone mirror object existing in the space.

In a second aspect, an example embodiment of the present applicationprovides an information processing apparatus, comprising:

an information acquisition module, configured to acquire sight lineinformation of at least one user in a space; and

a mirror object determination module, configured to determine, at leastaccording to the sight line information, at least one mirror objectexisting in the space.

In a third aspect, an example embodiment of the present applicationprovides a user equipment, the user equipment comprising:

a memory, configured to store an instruction;

a processor, configured to execute the instruction stored in the memory,the instruction causing the processor to perform the followingoperations of:

acquiring sight line information of at least one user in a space; and

determining, at least according to the sight line information, at leastone mirror object existing in the space.

In at least one of the example embodiments of the present application, amirror object in a space is determined according to sight lineinformation of at least one user, providing a basis formirror-object-based applications, for example, modeling the mirrorobject.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart of an information processing method according toan example embodiment of the present application;

FIGS. 2a-2b are schematic diagrams of application scenarios of aninformation processing method according to an example embodiment of thepresent application;

FIG. 3 is a schematic structural diagram of an information processingapparatus according to an example embodiment of the present application;

FIGS. 4a-4d are schematic structural diagrams of four informationprocessing apparatuses according to an example embodiment of the presentapplication; and

FIG. 5 is a schematic structural diagram of a user equipment accordingto an example embodiment of the present application.

DETAILED DESCRIPTION

Example embodiments of the present application are further describedbelow in detail with reference to the accompanying drawings (in whichlike elements are denoted by like reference numerals) and embodiments.The following embodiments are used for describing the presentapplication, but are not intended to limit the scope of the presentapplication.

A person skilled in the art may understand that the terms such as“first” and “second” in the present application are used only todifferentiate different steps, devices, modules, or the like, andneither represent any specific technical meaning, nor indicate anynecessary logical relationship between the terms.

A mirror object in the environment space may bring about difficulties tomodel reconstruction of the space, for example, if the mirror object isnot identified, it is considered that there is no object in the positionof the mirror object, or an image of an object in a mirror plane isconsidered as a real object, leading to incorrect reconstruction. A useror device may encounter various problems when using a model establishedso; therefore, as shown in FIG. 1, an embodiment of the presentapplication provides an information processing method, comprising:

S110 acquiring sight line information of at least one user in a space;and

S120 determining, at least according to the sight line information, atleast one mirror object existing in the space.

For example, an information processing apparatus provided in the presentapplication is used as an execution body of this embodiment, to performS110-S120. Specifically, the information processing apparatus may bedisposed in a user equipment in a manner of software, hardware, or acombination of software and hardware, or the information processingapparatus itself is the user equipment; the user equipment comprises,but is not limited to: a computer, a notebook, a tablet computer, asmartphone, smart glasses, and a smart helmet.

In the above example embodiment of the present application, a mirrorobject in a space is determined according to sight line information ofat least one user, providing a basis for mirror-object-basedapplications, for example, modeling the mirror object.

Each step of the embodiment of the present application is furtherdescribed through the following example embodiment.

S110 Acquire sight line information of at least one user in a space.

In some example embodiments, the sight line information comprises:

direction information, starting point position information and gazepoint position information of multiple sight lines of the at least oneuser in the space.

A person skilled in the art can know that, as sight lines of the usercannot focus on the air with no objects, each sight line of the user maycorrespond to an object. When a user watches an object, the position ofone eye of the user is a starting point position of a sight linecorresponding to the eye, the position on the object watched by the useris the gaze point position of the sight line, and the direction watchedby the eye is the direction of the sight line. In the embodiment of thepresent application, the direction of a sight line is the directionhaving a spatial position, for example, in the embodiment of the presentapplication, other sight line directions parallel to but not coincidingwith one sight line direction are considered to be sight line directionsdifferent from the sight line direction, or according to requirements ofprecision, it is feasible to set other sight line directions parallel toone sight line direction but having a distance outside a set errorrange, which are considered to be sight line directions different fromthe sight line direction.

In the embodiment of the present application, the at least one user maybe one user, and may also be multiple users.

For example, in one example embodiment, optionally, the multiple sightlines may be:

multiple sight lines corresponding to objects existing in the spacewatched by a user multiple times in the space after one or more periodsof time.

Alternatively, in one example embodiment, the multiple sight lines maybe:

multiple sight lines corresponding to objects existing in the spacewatched by multiple users multiple times in the space after one or moreperiods of time.

In the embodiment of the present application, the more the number of thesight lines corresponding to the sight line information is, and the moreevenly the sight lines are distributed, positions, shapes and the likeof the at least one mirror object can be determined more accurately. Insome example embodiments, as time goes on, the number of sight lines ofthe at least one user in the space may increase constantly, and thus thesight line information can be updated according to a set regularity. Forexample, the sight line information is updated constantly with a setcycle. In some example embodiments, an established model of the mirrorobject can be improved constantly through information corresponding tonew sight lines.

In one example embodiment, the multiple sight lines may be captured whenthe at least one user watches each object in the space unconsciously.Alternatively, in one example embodiment, it is also feasible toindicate the at least one user to intentionally watch the correspondingobject (for example, indicate the at least one user to watch the mirrorobject existing in the space), thus acquiring corresponding sight lineinformation.

In some possible application scenarios, gaze point positions of one ormore sight lines of the at least one user are not on a fixed object, forexample, a moving object may appear in the space, the at least one usermay watch the moving object, and thus in the embodiment of the presentapplication, in order to avoid noise brought about by noise sight linesfrom which such an object is watched, it is feasible to exclude thenoise sight lines first.

As presence of the moving object may be random or intermittent in thetime, it is feasible to determine such noise sight lines according toacquisition time information of the multiple sight lines, for example,if sight lines of a gaze point in the region only appear in a region ofthe space in one time period, such sight lines may be the noise sightlines. Therefore, in one example embodiment, the sight line informationfurther comprises:

acquisition time information of the multiple sight lines.

In one example embodiment, before the determining, at least according tothe sight line information, at least one mirror object, the methodfurther comprises:

excluding, according to the acquisition time information, at least onenoise sight line in the multiple sight lines.

In one example embodiment, S110 may acquire the sight line informationfrom at least one external device. For example, in one exampleembodiment, it is feasible to acquire the sight line information from adata server; or, in one example embodiment, it is feasible to acquirethe sight line information from at least one sight line tracking deviceof the at least one user and at least one locating apparatus.

The at least one sight line tracking device, for example, may be: asight line tracking device that respectively tracks sight linedirections of two eyes of a user to determine the position of a gazepoint relative to the user according to the sight line directions of thetwo eyes. Alternatively, the sight line tracking device may also be: asight line tracking device that tracks a sight line direction of one eyeof a user and focal length information of the eye, to obtain theposition of a gaze point relative to the user according to the sightline direction and the focal length information.

The at least one locating apparatus may be configured to acquireposition information of the at least one user in the space, thusobtaining starting point position information corresponding to the sightline.

In one example embodiment, for example, the at least one user is oneuser, at this point, the information processing apparatus, for example,may be a headset device of the user, the headset device may capturesight line information of the user, and at this point, S110, forexample, may be capturing the sight line information.

S120 Determine, at least according to the sight line information, atleast one mirror object existing in the space.

In the following description of the embodiment of the presentapplication, matching between a point and an object surface, forexample, may be that a position difference between the point and theobject surface is within a set deviation range.

When a mirror object is present in the space, in a sight line direction,the user may focus on the mirror object and may also focus on objectsbehind the mirror object, and thus in one example embodiment, S120comprises:

in response to a determination that, according to the sight lineinformation, a first gaze point and a second gaze point are distributedfront and back in a sight line direction, determining that there is amirror object in the position passing through the first gaze point inthe sight line direction, wherein two sight line starting points of twosight lines respectively comprising the first gaze point and the secondgaze point are both before the first gaze point, and the second gazepoint does not match a surface of an object.

In one example embodiment, only when two sight line directions of twosight lines are both completely the same as the sight line direction,can the first gaze point and the second gaze point be considered to bedistributed front and back in the sight line direction; in other exampleembodiments, according to different precision requirements, it is alsofeasible to consider that the two sight line directions are the same asthe sight line direction when the difference between the two sight linedirections of the two sight lines and the sight line direction is withinthe set range, that is, the first gaze point and the second gaze pointare in the sight line direction.

In the embodiment of the present application, front and back arerelative to the sight line direction, for example, an upstream positionin the sight line direction is before a downstream position in the sightline direction. In the embodiment of the present application, in thesight line direction, the first gaze point is upstream of the secondgaze point.

In one possible application scenario, in one sight line direction, onesight line may be in a front side of an object, and a corresponding gazepoint position thereof is on the object; the other sight line may be ina back side of the object, and a corresponding gaze point positionthereof is on another object behind the object. At this point, althoughthe object may not be a mirror object, a first gaze point and a secondgaze point distributed front and back may still be present in the sightline direction. Therefore, in the embodiment of the present application,in order to exclude such a situation and improve accuracy ofdetermination of the mirror object, it is required that two sight linestarting points of two sight lines respectively comprising the firstgaze point and the second gaze point are both before the first gazepoint. In this way, when an object is an ordinary object (for example,the object is neither a transparent object nor a mirror object), as theuser's sight line cannot penetrate the object to reach the back side ofthe object, the situation may not occur where the starting point is in afront side of a gaze point in the sight line direction and the gazepoint is a sight line in a rear side of the gaze point.

In one possible application scenario, when the first gaze point and thesecond gaze point distributed front and back are in a sight linedirection and starting points of the two sight lines are both before thefirst gaze point, in addition to the mirror object, it is also likelythat the object corresponding to the first gaze point is a transparentobject, and at this point, in the sight line direction, the user's sightline may fall on the transparent object and may also fall on an objectbehind the transparent object. Therefore, in some example embodiments,it is also feasible to determine that the first gaze point is on amirror object by determining that the first gaze point is not on atransparent object. Thus, in the embodiment of the present application,it is required that the second gaze point does not match an objectsurface. In this way, the second gaze point is not a gaze point at whichthe object behind the first gaze point is gazed, and thus the first gazepoint is a gaze point at which the mirror object is gazed.

In the example embodiment, that the second gaze point does not match theobject surface, for example, may be that the position difference betweenthe second gaze point and the object surface is outside a set deviationrange. Herein, it is feasible to first suppose that an object gazed bythe first gaze point is a transparent object, as light may be refractedor shifted when passing through a general transparent object, thedeviation range, for example, may be determined according to at leastone attribute of the transparent object, where the at least oneattribute, for example, may be the shape, transmissivity and the like ofthe transparent object. When the position difference between the secondgaze point and the object surface is outside the deviation range, it isfeasible to consider that the object gazed by the first gaze point isnot the transparent object, but a mirror object.

In one example embodiment, for example, it is possible to determine,according to gaze point positions in the sight line informationcorresponding to the object surface, the object surface and the positionthereof. In other example embodiments, it is also possible to determinethe object surface and the position thereof with another automaticmodeling method, for example, an image of the place of the object isacquired, and the object surface and the position thereof are obtainedwith an image processing method.

In one example embodiment, after it is determined that the mirror objectis present in a sight line direction, the method may further comprise:determining, at least according to the sight line information, a regionwhere the mirror object is. In this way, it is possible to provide datafor modeling of the mirror object.

In one example embodiment, the determining, at least according to thesight line information, a region where the mirror object is maycomprise:

obtaining, according to the sight line information, position informationof all first gaze points in all sight line directions with a first gazepoint and a second gaze point; and

determining, according to distribution regularities of the all firstgaze points and position of the first gaze point in the sight linedirection, the region where the mirror object is.

In the example embodiment, each sight line direction in the all sightline directions has a first gaze point and a second gaze pointdistributed front and back, starting points of two sight linescorresponding to the two gaze points are both before the first gazepoint, and the position of the second gaze point does not match asurface of an object.

In one example embodiment, after positions of the all first gaze pointsare obtained, as the all first gaze points may not be on the same mirrorobject, it is feasible to classify the all first gaze points accordingto distribution regularities of the all first gaze points, causing firstgaze points from which one mirror object is gazed to be distinguishedfrom first gaze points from which another mirror object is gazed. Forexample, the all first gaze points are converted to point cloud, eachfirst gaze point is a corresponding point in the point cloud, and thenpoints corresponding to different mirror objects are distinguished witha point cloud segmentation method. A region corresponding to a mirrorobject can be determined according to multiple first gaze pointscorresponding to the mirror object, for example, the region is obtainedaccording to positions of the multiple first gaze points with a surfacereconstruction method, for example, a Poisson surface reconstructionmethod. Definitely, other possible surface reconstruction methods mayalso be applied to the embodiment of the present application.

Through the above example embodiment, the region drawn by a first gazepoint set, where a first gaze point in a sight line direction is, is theregion where the mirror object corresponding to the sight line directionis.

Definitely, in other example embodiments, after it is determined that afirst gaze point and a second gaze point are in a sight line direction,it is feasible to look for whether there are other sight line directionswith a first gaze point and a second gaze point in a set range near thefirst gaze point. In this way, a sight line direction set may also beobtained, and the region where the mirror object corresponding to thesight line direction is may also be obtained according to the first gazepoint set corresponding to the sight line direction set.

In another example embodiment, the determining, at least according tothe sight line information, at least one mirror object comprises:

in response to a determination that, according to the sight lineinformation, there is at least one set region in the space, determiningthat there is at least one mirror object in the at least one set region;

wherein, for each sight line direction towards a set region in the atleast one set region:

there is a first gaze point and a second gaze point distributed frontand back;

wherein the first gaze point is located in the set region, and thesecond gaze point does not match an object surface behind the setregion.

In the example embodiment, a mirror object is determined not through aseparate sight line direction, but multiple sight line directions whichare obtained by analyzing all sight lines corresponding to the sightline information. Wherein there are a first gaze point and a second gazepoint existing in each sight line direction in the multiple sight linedirections, at least one region is obtained according to positions ofall first gaze points corresponding to the multiple sight linedirections, then the at least one region is verified, and whether, foreach sight line direction towards a region in the at least one region,there is a first gaze point and a second gaze point distributed frontand back as described above, if yes, the at least one region is the atleast one set region, and if no, it is necessary to re-determine the atleast one region, until the at least one set region is obtained.

It can be seen that, when the sight line information corresponds to moresight lines, the at least one set region may be determined moreaccurately.

In some possible application scenarios, for example, after a mirrorobject is determined and when the position of at least one objectsurface that a mirror plane of a mirror object in the set region facescannot be determined, it is feasible to determine the position of the atleast one object surface according to the position of the second gazepoint in each sight line direction corresponding to the mirror object.For example, in one example embodiment, optionally, the method mayfurther comprise:

obtaining, according to the sight line information, position informationof all second gaze points in all sight line directions with a first gazepoint and a second gaze point; and

determining, according to distribution regularities of the all secondgaze points, the position of the at least one object surface.

In some example embodiments, when it is impossible to determine whethera second gaze point corresponds to an object surface (for example,positions of objects behind the mirror object still cannot bedetermined), at this point, it is also feasible to determine whether aregion of a first gaze point has a mirror object by determining whetherthe second gaze point has a mirror symmetrical object relative to anobject where the corresponding first gaze point is. At this point, S120may comprise:

in response to a determination that, according to the sight lineinformation, there is at least one set region in the space, determiningthat there is at least one mirror object in the at least one set region;

wherein, for each sight line direction towards a set region in the atleast one set region:

there is a first gaze point and a second gaze point distributed frontand back;

wherein the first gaze point is located in the set region, and asymmetry point which is symmetric with the second gaze point withrespect to a surface of the set region matches a surface of an object.

In the example embodiment, when the symmetry point matches the surfaceof the object, it is possible to consider that the first gaze point ison a mirror object.

In the example embodiment, it is possible to determine at least oneregion according to all first gaze points corresponding to all gazedirections comprised in the gaze information, wherein for each gazedirection of the all gaze directions, there are a first gaze point and asecond gaze point distributed front and back and starting points of twosight lines corresponding to the first gaze point and the second gazepoint are both before the first gaze point. And then verify all secondgaze points corresponding to the all gaze directions according to the atleast one region, to determine whether symmetry points related to theall second gaze points match at least one surface of at least oneobject; if yes, the at least one region is the at least one set region;and if no, it is further necessary to exclude a first gaze pointcorresponding to a second gaze point that a corresponding symmetry pointdoesn't match a surface of an object from the all first gaze points, tore-construct at least one new region, until all the at least one regionis in line with the definition of the at least one set region as above.

In the example embodiment, the position of the object surface may alsobe determined through the sight line information.

The embodiment of the present application is further described throughthe following application scenario.

In one possible scenario, sight line information corresponding to aspace comprises multiple sight lines as shown in FIG. 2a , wherein eachsight line is represented with an arrowed straight line; definitely, forreadability of the drawing, FIG. 2a merely exemplarily illustrates somesight lines.

In the following description of the example embodiment, for a situationwhere a sight line direction has two gaze points distributed front andback, the former gaze point is in the back of starting points of the twogaze points and the latter gaze point does not match an object surface,the former gaze point is referred to as first gaze point, and the lattergaze point is referred to as second gaze point. In a situation where asight line direction has a gaze point or has multiple gaze points notsatisfying the above situation, the gaze point(s) is/are referred to asthird gaze point.

In the example embodiment, according to the sight line information, itis obtained that a first sight line direction Ea has a first gaze point211 a and a second gaze point 221 a distributed front and back along thefirst sight line direction Ea, wherein the first gaze point 211 a is agaze point of a first sight line 210, the second gaze point 221 a is agaze point of a second sight line 220, and a first starting point 212 ofthe first sight line 210 and a second starting point 222 of the secondsight line are both in a front side of the first gaze point 211 a.

In the example embodiment, according to the sight line information, itcan be obtained that there is no third gaze point near the position ofthe second gaze point 221 a, and thus it is feasible to consider thatthe second gaze point 221 a does not match a surface of any object.

It can be seen from the above that there is a mirror object located inthe first sight line direction Ea, and the first gaze point 211 a is onthe mirror object.

Further, in order to determine a region where the mirror object is, inthe example embodiment, it is also feasible to find all sight linedirections with a first gaze point and a second gaze point. In theexample embodiment, as shown in FIG. 2a , several such sight linedirections are further exemplarily illustrated: a third sight linedirection Ec, a fourth sight line direction Ed, a fifth sight linedirection Ef and a sixth sight line direction Ef, and a person skilledin the art can know that many other such sight line directions are notillustrated in FIG. 2a . According to positions of the first gaze points211 a, 211 c-211 f in the all sight line directions, it is feasible todetermine the region where the mirror object 240 is, as shown in FIG. 2b(a person skilled in the art can know that, in order to obtain theregion of the mirror object 240 in FIG. 2b , in addition to the first,third to sixth sight line directions described as above, other sightline directions with a first gaze point not illustrated are also used).

In the example embodiment, it is also feasible to determine an image250′ of an object in the mirror object 240 according to the second gazepoints 221, 221 c-221 f in the first, third to sixth sight linedirections.

In another example embodiment, in addition to determining that the firstgaze points 211 a, 211 c-211 f correspond to the mirror object 240according to that the second gaze points 221, 221 c-221 f do not fall onan object, it is also feasible to determine that the regioncorresponding to the first gaze points 211 a, 211 c-211 f corresponds tothe mirror object 240 according to that mirror points 221 a′, 221 c′-221f of the second gaze points 221, 221 c-221 f with respect to the regionsfall on an object.

For example, a second sight line direction Eb has a third sight line230, and a distance difference between a third sight line point 231 ofthe third sight line 230 and the mirror point 221 a′ of the second gazepoint 221 a of the second sight line 220 is within a set deviationrange, which basically coincide with each other. Herein, presence of thethird sight line point 231 indicates that at the third sight line point231 is a surface of an object, and thus it is feasible to consider thatthe mirror point 221 a′ of the second gaze point 221, symmetricalrelative to the surface of the region of the first gaze point 211 a,matches a surface of an object.

In the example embodiment, in addition to the mirror object 240, it isalso feasible to determine positions of other objects according to thesight line information. For example, according to position informationof all third gaze points in the sight line information, it is feasibleto obtain that the space further has the first object 250 and the secondobject 260 as shown in FIG. 2 b.

It should be understood by a person skilled in the art that, in any oneof the foregoing methods of the specific implementations of the presentapplication, the value of the serial number of each step described abovedoes not mean an execution sequence, and the execution sequence of eachstep should be determined according to the function and internal logicthereof, and should not be any limitation to the implementationprocedure of the specific implementations of the present application.

As shown in FIG. 3, an embodiment of the present application provides aninformation processing apparatus 300, comprising:

an information acquisition module 310, configured to acquire sight lineinformation of at least one user in a space; and

a mirror object determination module 320, configured to determine, atleast according to the sight line information, at least one mirrorobject existing in the space.

In the above example embodiment of the present application, a mirrorobject in a space is determined according to sight line information ofat least one user, providing a basis for mirror-object-basedapplications, for example, modeling the mirror object.

The embodiment of the present application is further described throughthe following example embodiment.

In some example embodiments, the sight line information comprises:

direction information, starting point position information and gazepoint position information of multiple sight lines of the at least oneuser in the space. Reference can be made to the correspondingdescription in the embodiments shown in FIG. 1 and FIGS. 2a-2b for thedescription about the information.

In the embodiment of the present application, the at least one user maybe one user, and may also be multiple users.

For example, in one example embodiment, optionally, the multiple sightlines may be:

multiple sight lines corresponding to objects existing in the spacewatched by a user multiple times in the space after one or more periodsof time.

Alternatively, in one example embodiment, the multiple sight lines maybe:

multiple sight lines corresponding to objects existing in the spacewatched by multiple users multiple times in the space after one or moreperiods of time.

In the embodiment of the present application, the more the number of thesight lines corresponding to the sight line information is, and the moreevenly the sight lines are distributed, positions, shapes and the likeof the at least one mirror object can be determined more accurately. Insome example embodiments, as time goes on, the number of sight lines ofthe at least one user in the space may increase constantly, and thus thesight line information can be updated according to a set regularity. Forexample, the sight line information is updated constantly with a setcycle. In some example embodiments, an established model of the mirrorobject can be improved constantly through information corresponding tonew sight lines.

In one example embodiment, the multiple sight lines may be captured whenthe at least one user watches each object in the space unconsciously.Alternatively, in one example embodiment, it is also feasible toindicate the at least one user to intentionally watch the correspondingobject (for example, indicate the at least one user to watch the mirrorobject existing in the space, especially a boundary position of themirror object), thus acquiring corresponding sight line information.

In some possible application scenarios, gaze point positions of one ormore sight lines of the at least one user are not on a fixed object, forexample, a moving object may appear in the space, the at least one usermay watch the moving object, and thus in the embodiment of the presentapplication, in order to avoid noise brought about by noise sight linesfrom which such an object is watched, it is feasible to exclude thenoise sight lines first.

As presence of the moving object may be random or intermittent in thetime, it is feasible to determine such noise sight lines according toacquisition time information of the multiple sight lines, for example,if sight lines of a gaze point in the region only appear in a region ofthe space in one time period, such sight lines may be the noise sightlines. Therefore, in one example embodiment, the sight line informationfurther comprises:

acquisition time information of the multiple sight lines.

As shown in FIG. 4a , in the example embodiment, the apparatus 300further comprises:

a noise excluding module 330, configured to exclude, according to theacquisition time information, at least one noise sight line in themultiple sight lines.

The at least one noise sight line comprises: at least one sight linefrom which a moving object is watched.

In one example embodiment, the information acquisition module 310 maycomprise multiple sensors, configured to capture the sight lineinformation. The sensors, for example, may comprise a locator thatlocates the position of at least one user in a space, and a sight linetracking device that captures sight line directions of the at least oneuser and positions of gaze points relative to the user.

In another example embodiment, the information acquisition module 310may comprise a communication device, configured to acquire the sightline information from at least one external device. The at least oneexternal device, for example, may comprise a device that comprises theabove multiple sensor, or may be at least one data server that collectsthe sight line information.

As shown in FIG. 4b , in one example embodiment, the mirror objectdetermination module 320 comprises:

a first determination unit 321, configured to, in response to adetermination that, according to the sight line information, a firstgaze point and a second gaze point are distributed front and back in asight line direction, determine that there is a mirror object in theposition passing through the first gaze point in the sight linedirection;

wherein two sight line starting points of two sight lines respectivelycomprising the first gaze point and the second gaze point are bothbefore the first gaze point, and the second gaze point does not match anobject surface.

In one example embodiment, only when two sight line directions of twosight lines are both completely the same as the sight line direction,can the first gaze point and the second gaze point be considered to bedistributed front and back in the sight line direction; in other exampleembodiments, according to different precision requirements, it is alsofeasible to consider that the two sight line directions are the same asthe sight line direction when the difference between the two sight linedirections of the two sight lines and the sight line direction is withinthe set range, that is, the first gaze point and the second gaze pointare in the sight line direction.

In the embodiment of the present application, front and back arerelative to the sight line direction.

In one possible application scenario, when the first gaze point and thesecond gaze point distributed front and back are in a sight linedirection and starting points of the two sight lines are both before thefirst gaze point, in addition to the mirror object, it is also likelythat the object corresponding to the first gaze point is a transparentobject, and at this point, in the sight line direction, the user's sightline may fall on the transparent object and may also fall on an objectbehind the transparent object. Therefore, in some example embodiments,it is also feasible to determine that the first gaze point is on amirror object by determining that the first gaze point is not on atransparent object. Thus, in the embodiment of the present application,it is required that the second gaze point does not match an objectsurface. In the example embodiment, that the second gaze point does notmatch the object surface, for example, may be that the positiondifference between the second gaze point and the object surface isoutside a set deviation range.

In one example embodiment, after it is determined that the mirror objectis present in a sight line direction, it may be further necessary todetermine a region where the mirror object is, which, for example,provides data for modeling of the mirror object. Therefore, in theexample embodiment, the apparatus 300 further comprises:

a region determination module 340, configured to determine, at leastaccording to the sight line information, a region where the mirrorobject is.

In one example embodiment, optionally, the region determination module340 may comprise:

a first position information acquisition unit 341, configured to obtain,according to the sight line information, position information of allfirst gaze points in all sight line directions with a first gaze pointand a second gaze point.

In the example embodiment, each sight line direction in the all sightline directions has a first gaze point and a second gaze point, startingpoints of two sight lines corresponding to the two gaze points are bothbefore the first gaze point, and the position of the second gaze pointdoes not match an object surface.

a first position information processing unit 342, configured todetermine, according to distribution regularities of the all first gazepoints and position of the first gaze point in the sight line direction,the region where the mirror object is.

In one example embodiment, after the position of the all first gazepoint are obtained, as it is likely that the all first gaze points arenot all on the same mirror object, it is feasible to classify the allfirst gaze points according to distribution regularities of the allfirst gaze points, causing first gaze points from which one mirrorobject is gazed to be distinguished from first gaze points from whichanother mirror object is gazed.

Through the above example embodiment, the region drawn by a first gazepoint set, where a first gaze point in a sight line direction is, is theregion where the mirror object corresponding to the sight line directionis.

Definitely, in other example embodiments, after it is determined that afirst gaze point and a second gaze point are in a sight line direction,it is feasible to look for whether there are other sight line directionswith a first gaze point and a second gaze point in a set range near thefirst gaze point. In this way, a sight line direction set may also beobtained, and the region where the mirror object corresponding to thesight line direction is may also be obtained according to the first gazepoint set corresponding to the sight line direction set.

In another example embodiment, as shown in FIG. 4c , the mirror objectdetermination module 320 may comprise:

a third determination unit 323, configured to, in response to adetermination that, according to the sight line information, there is atleast one set region in the space, determine that there is at least onemirror object in the at least one set region;

wherein, for each sight line direction towards a set region in the atleast one set region:

there is a first gaze point and a second gaze point distributed frontand back;

wherein the first gaze point is located in the set region, and thesecond gaze point does not match an object surface behind the setregion.

Reference can be made to the corresponding description in the embodimentshown in FIG. 1 for further description about the implementation of thefunction of the third determination unit 323.

In the solution of the first determination unit 321 and the thirddetermination unit 323, it is necessary to know object surface positioninformation behind the mirror object. In one example embodiment, asshown in FIG. 4c , the apparatus 300 further comprises:

a second object position determination module 350, configured todetermine, according to the sight line information, the position of theobject surface. For example, in the example embodiments shown in FIGS.2a-2b , the position of the object surface is obtained throughrespective third gaze points.

In some possible application scenarios, for example, after a mirrorobject is determined and when positions of objects behind the mirrorobject cannot be determined, it is feasible to determine the positionsof the objects according to the position of the second gaze point inreach sight line direction corresponding to the mirror object.Therefore, in one example embodiment, optionally, as shown in FIG. 4c ,the apparatus 300 further comprises:

a first object position determination module 360, configured todetermine, at least according to the gaze information, the position ofat least one object surface that a mirror plane of a mirror object inthe set region faces.

In one example embodiment, the first object position determinationmodule 360 comprises:

a second position information acquisition unit 361, configured toobtain, according to the sight line information, position information ofall second gaze points in all sight line directions with a first gazepoint and a second gaze point; and

a second position information processing unit 362, configured todetermine, according to distribution regularities of the all second gazepoints, the position of the at least one object surface.

In some example embodiments, when it is impossible to determine whethera second gaze point corresponds to an object surface (for example,positions of objects behind the mirror object still cannot bedetermined), at this point, it is also feasible to determine whether aregion of a first gaze point has a mirror object by determining whetherthe second gaze point has a mirror symmetrical object relative to anobject where the corresponding first gaze point is. Therefore, in oneexample embodiment, as shown in FIG. 4d , the mirror objectdetermination module 320 may comprise:

a second determination unit 322, configured to, in response to adetermination that, according to the sight line information, there is atleast one set region in the space, determine that there is at least onemirror object in the at least one set region;

wherein, for each sight line direction towards a set region in the atleast one set region:

there is a first gaze point and a second gaze point distributed frontand back;

wherein the first gaze point is located in the set region, and asymmetry point which is symmetric with the second gaze point withrespect to a surface of the set region matches a surface of an object.

When the symmetry point matches the surface of the object, it isfeasible to consider that the first gaze point is on a mirror object,because the second gaze point corresponds to a mirror image of thesurface of the object in the mirror object; and when the symmetry pointdoes not match the object surface, it is feasible to consider that thefirst gaze point is on a transparent object.

Reference can be made to the corresponding description in the embodimentshown in FIG. 1 for further description of the implementation of thefunction of the second determination unit 322.

Reference can be made to the corresponding description in the abovemethod embodiment for the further description about the implementationof the functions of the modules and units in the embodiment of thepresent application.

FIG. 5 is a schematic structural diagram of a user equipment 500according to an embodiment of the present application; the specificembodiment of the present application does not limit specificimplementation of the user equipment 500. As shown in FIG. 5, the userequipment 500 may comprise:

a processor 510, a communications interface 520, a memory 530, and acommunications bus 540.

The processor 510, the communications interface 520, and the memory 530communicate with each other by using the communications bus 540.

The communications interface 520 is configured to communicate with anetwork element such as a client.

The processor 510 is configured to execute an instruction 532, andspecifically, may implement relevant steps in the above methodembodiments.

Specifically, the instruction 532 may comprise a computer operationinstruction.

The processor 510 may be a central processing unit (CPU), or anapplication specific integrated circuit (ASIC), or one or moreintegrated circuits configured to implement the embodiments of thepresent application.

The memory 530 is configured to store the instruction 532. The memory530 may comprise a high-speed random access memory (RAM), or may alsocomprise a non-volatile memory, for example, at least one magnetic diskmemory. The instruction 532 may be specifically configured to enable theuser equipment 500 to perform the following steps:

acquiring sight line information of at least one user in a space; and

determining, at least according to the sight line information, at leastone mirror object existing in the space.

For specific implementation of the steps in the instruction 532,reference may be made to corresponding description in the correspondingsteps and units in the embodiments, and no further details are providedherein again. A person skilled in the art may clearly know that, for thepurpose of convenient and brief description, for a detailed workingprocess of the foregoing device and modules, reference may be made to acorresponding process in the foregoing method embodiments, and nofurther details are provided herein again.

A person of ordinary skill in the art may be aware that, with referenceto the examples described in the embodiments disclosed in thisspecification, units and algorithm steps may be implemented byelectronic hardware, or a combination of computer software andelectronic hardware. Whether the functions are performed by hardware orsoftware depends on particular applications and design constraintconditions of the technical solutions. A person skilled in the art mayuse different methods to implement the described functions for eachparticular application, but it should not be considered that theimplementation goes beyond the scope of the present application.

When the functions are implemented in a form of a software functionalunit and sold or used as an independent product, the functions may bestored in a computer-readable storage medium. Based on such anunderstanding, the technical solutions of the present applicationessentially, or the part contributing to the prior art, or a part of thetechnical solutions may be implemented in a form of a software product.The computer software product is stored in a storage medium, andcomprises several instructions for instructing a computer device (whichmay be a personal computer, a server, a network device, or the like) toperform all or a part of the steps of the methods described in theembodiments of the present application. The foregoing storage mediumcomprises: any medium that can store program code, such as a USB flashdrive, a removable hard disk, a read-only memory (ROM, Read-OnlyMemory), a RAM, a magnetic disk, or an optical disc.

The foregoing example embodiments are merely used for describing thepresent application, rather than limiting the present application. Aperson of ordinary skill in the art may made various changes andmodifications without departing from the spirit and scope of the presentapplication, and therefore, all equivalent technical solutions shallbelong to the scope of the present application, and the protection scopeof the present application shall be subject to the claims.

What is claimed is:
 1. A method, comprising: acquiring, by a systemcomprising a processor, sight line information of the at least one userin a space, wherein the sight line information is based upon multiplesight lines of the at least one user in the space and includesacquisition time information of the multiple sight lines; excluding, bythe system according to the acquisition time information of the multiplesight lines, at least one noise sight line in the multiple sight lines,wherein the at least one noise sight line comprises at least one sightline from which a moving object is being watched; and determining, bythe system—at least according to the sight line information, at leastone mirror object existing in the space.
 2. The method of claim 1,wherein the determining, at least according to the sight lineinformation, the at least one mirror object comprises: in response to adetermination that, according to the sight line information, a firstgaze point and a second gaze point are distributed front and back in asight line direction, determining that a mirror object of the at leastone mirror object exists in a first position passing through the firstgaze point in the sight line direction, wherein two sight line startingpoints of two sight lines respectively comprising the first gaze pointand the second gaze point are before the first gaze point, and whereinthe second gaze point does not match a surface of an object.
 3. Themethod of claim 2, further comprising: determining, at least accordingto the sight line information, a region where the mirror object islocated.
 4. The method of claim 3, wherein the determining, at leastaccording to the sight line information, the region where the mirrorobject is located comprises: obtaining, according to the sight lineinformation, position information of a plurality of gaze points in aplurality of sight line directions with the first gaze point and thesecond gaze point; and determining, according to distributionregularities of the plurality of the gaze points and a second positionof the first gaze point in the sight line direction, the region wherethe mirror object is located.
 5. The method of claim 2, furthercomprising: determining, according to the sight line information, asecond position of the surface of the object.
 6. The method of claim 1,wherein the determining, at least according to the sight lineinformation, the at least one mirror object comprises: in response to adetermination that, according to the sight line information, there is atleast one set region in the space, determining that there is the atleast one mirror object in the at least one set region; and wherein, foreach sight line direction towards a set region in the at least one setregion: there is a first gaze point and a second gaze point distributedfront and back, wherein the first gaze point is located in the setregion, and wherein a symmetry point, which is symmetric with the secondgaze point with respect to a surface of the set region, matches asurface of an object.
 7. The method of claim 1, wherein the determining,at least according to the sight line information, the at least onemirror object comprises: in response to a determination that, accordingto the sight line information, there is at least one set region in thespace, determining that there is the at least one mirror object in theat least one set region; and wherein, for each sight line directiontowards a set region in the at least one set region: there is a firstgaze point and a second gaze point distributed front and back, whereinthe first gaze point is located in the set region, and wherein thesecond gaze point does not match a surface of an object behind the setregion.
 8. The method of claim 7, further comprising: determining, atleast according to gaze information, a second position of at least onesurface of at least one object that a mirror plane of a mirror object ofthe at least one mirror object in the set region faces.
 9. The method ofclaim 8, wherein the determining, at least according to the gazeinformation, the second position of the at least one surface of the atleast one object comprises: obtaining, according to the sight lineinformation, position information of a plurality of gaze points in aplurality of sight line directions with the first gaze point and thesecond gaze point; and determining, according to distributionregularities of the plurality of the gaze points, the second position ofthe at least one surface of the at least one object.
 10. The method ofclaim 1, wherein the at least one user comprises multiple users.
 11. Themethod of claim 1, wherein the sight line information comprises:direction information, starting point position information and gazepoint position information of the multiple sight lines of the at leastone user in the space.
 12. An apparatus, comprising: a memory thatstores executable modules; and a processor, coupled to the memory, thatexecutes or facilitates execution of the executable modules, theexecutable modules comprising: an information acquisition moduleconfigured to acquire sight line information, wherein the sight lineinformation is based upon multiple sight lines of at least one user in aspace; a noise excluding module configured to exclude, according to theacquisition time information, a noise sight line in the multiple sightlines, wherein the noise sight line comprises a sight line from which amoving object is being watched; and a mirror object determination moduleconfigured to determine, at least according to the sight lineinformation, a mirror object existing in the space.
 13. The apparatus ofclaim 12, wherein the mirror object determination module comprises: afirst determination unit configured to, in response to a determinationthat, according to the sight line information, a first gaze point and asecond gaze point are distributed front and back in a sight linedirection, determine the mirror object is in a position passing throughthe first gaze point in the sight line direction, wherein two sight linestarting points of two sight lines respectively comprising the firstgaze point and the second gaze point are before the first gaze point,and wherein the second gaze point does not match a surface of an object.14. The apparatus of claim 13, wherein the executable modules furthercomprise: a region determination module configured to determine, atleast according to the sight line information, a region where the mirrorobject is located.
 15. The apparatus of claim 14, wherein the positionpassing through the first gaze point is a first position, and whereinthe region determination module comprises: a position informationacquisition unit configured to obtain, according to the sight lineinformation, position information of a plurality of gaze points in aplurality of sight line directions with the first gaze point and thesecond gaze point; and a position information processing unit configuredto determine, according to distribution regularities of the plurality ofthe gaze points and a second position of the first gaze point in thesight line direction, the region where the mirror object is located. 16.The apparatus of claim 13, wherein the position passing through thefirst gaze point is a first position, and wherein the executable modulesfurther comprise: an object position determination module configured todetermine, according to the sight line information, a second position ofa surface of an object.
 17. The apparatus of claim 12, wherein themirror object determination module comprises: a determination unitconfigured to, in response to a determination that, according to thesight line information, there is the set region in the space, determinethat there is the mirror object in the set region; and wherein, for eachsight line direction towards the set region: there is a first gaze pointand a second gaze point distributed front and back, wherein the firstgaze point is located in the set region, and wherein a symmetry point,which is symmetric with the second gaze point with respect to a surfaceof the set region, matches a surface of an object.
 18. The apparatus ofclaim 12, wherein the mirror object determination module comprises: adetermination unit configured to, in response to a determination that,according to the sight line information, there is the set region in thespace, determine that there is the mirror object in the set region; andwherein, for each sight line direction towards the set region in the setregion: there is a first gaze point and a second gaze point distributedfront and back, wherein the first gaze point is located in the setregion, and wherein the second gaze point does not match a surface of anobject behind the set region.
 19. The apparatus of claim 18, wherein theposition passing through the first gaze point is a first position, andwherein the executable modules further comprise: an object positiondetermination module configured to determine, at least according to gazeinformation, a second position of the surface of the object that amirror plane of the mirror object in the set region faces.
 20. Theapparatus of claim 19, wherein the object position determination modulecomprises: a position information acquisition unit configured to obtain,according to the sight line information, position information of aplurality of gaze points in a plurality of sight line directions withthe first gaze point and the second gaze point; and a positioninformation processing unit configured to determine, according todistribution regularities of the plurality of the gaze points, thesecond position of the surface of the object.
 21. The apparatus of claim12, wherein the sight line information is sight line information ofmultiple users, comprising the at least one user.
 22. The apparatus ofclaim 12, wherein the sight line information comprises: directioninformation, starting point position information, and gaze pointposition information of the multiple sight lines of the at least oneuser in the space.
 23. A user equipment, wherein the user equipmentcomprises: a sensor device; a memory configured to store at least oneinstruction; a processor, configured to execute the instruction storedin the memory, the at least one instruction causing the processor toperform operations, comprising: generating sight line information of theat least one user in the space, wherein the sight line information isbased upon multiple sight lines of the at least one user in the spaceand includes acquisition time information of the multiple sight lines;excluding, according to the acquisition time information of the multiplesight lines, at least one noise sight line in the multiple sight lines,wherein the at least one noise sight line comprises at least one sightline from which a moving object is being watched; and at least based onthe sight line information, determining at least one mirror objectexisting in the space.
 24. The user equipment of claim 23, wherein thedetermining the at least one mirror object comprises: in response to adetermination that, according to the sight line information, a firstgaze point and a second gaze point are distributed front and back in asight line direction, determining that a mirror object of the at leastone mirror object exists in a first position passing through the firstgaze point in the sight line direction, wherein two sight line startingpoints of two sight lines respectively comprising the first gaze pointand the second gaze point are before the first gaze point, and whereinthe second gaze point does not match a surface of an object.
 25. Theuser equipment of claim 24, the operations further comprising:determining, at least according to the sight line information, a regionwhere the mirror object is located.
 26. The user equipment of claim 25,wherein the determining, at least according to the sight lineinformation, the region where the mirror object is located comprises:obtaining, according to the sight line information, position informationof a plurality of gaze points in a plurality of sight line directionswith the first gaze point and the second gaze point; and determining,according to distribution regularities of the plurality of the gazepoints and a second position of the first gaze point in the sight linedirection, the region where the mirror object is located.
 27. The userequipment of claim 24, wherein the determining the at least one mirrorobject comprises: in response to a determination that, according to thesight line information, there is at least one set region in the space,determining that there is the at least one mirror object in the at leastone set region; and wherein, for each sight line direction towards a setregion in the at least one set region: there is a first gaze point and asecond gaze point distributed front and back, wherein the first gazepoint is located in the set region, and wherein a symmetry point, whichis symmetric with the second gaze point with respect to a surface of theset region, matches a surface of an object.
 28. The user equipment ofclaim 24, wherein the determining the at least one mirror objectcomprises: in response to a determination that, according to the sightline information, there is at least one set region in the space,determining that there is the at least one mirror object in the at leastone set region; and wherein, for each sight line direction towards a setregion in the at least one set region: there is a first gaze point and asecond gaze point distributed front and back, wherein the first gazepoint is located in the set region, and wherein the second gaze pointdoes not match a surface of an object behind the set region.
 29. Theuser equipment of claim 28, the operations further comprising:determining, at least according to gaze information, a second positionof at least one surface of at least one object that a mirror plane of amirror object of the at least one mirror object in the set region faces.